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INTRODUCTION 


The developnent of the Vertical Motion Simulator (VMS) 
has been progressing steadily and will soon be complete. 

A desire to gauge the success of the VMS development 
effort has lead to questions about how to measure and 
evaluate VMS performance capabilities and what those 
performance capabilities should be. As a response to 
those questions, the following document has been created. 

The VMS is an aircraft simulator designed to simulate a 
variety of experimental helicopter and STOL/VTOL aircraft 
and can be adapted to simulate other kinds of aircraft 
with special pitch and Z axis characteristics. The VMS 
will include a large motion base with extensive vertical 
and lateral travel capabilities, a computer generated 
image (CGI) visual system, and a high speed CDC 7600 
computer system, which will perform aero model calculations 

Developing guidelines on how to measure and evaluate VMS 
performance has been complicated. No clear cut approach 
was obvious and few relevant documents were available'. 

As a first step, a survey of simulation users was conducted 
Appropriate personnel in the airlines, the FAA, and the 
military were questioned concerning how they evaluated and 
certified simulators for use (reference Appendix A for a 
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list eoBtscts). Tlie results of thst survey csa be found 
in the sections: ^Sinulstor Certification, Federal Aviation 
Adninistration and the Cosmercial Airlines *' and '’Simulator 
Certification, U.S. Air Force, Army, and Havy." *nie general 
outcome of the survey was that simulator certification 
primarily involves assessing simulator fidelity and that 
what fidelity really means can (and should) vary according 
to user needs. That outcome and its implications are 
discussed in the section: "Summary and Implications." 

It was becoming quite obvious that VUS user needs would be 
relevant to how VMS pe!k*formance should be measured and 
evaluated. In order to clarify and define what those 
user needs really were, a survey of VMS users was conducted. 
The results are discussed in the section: "Survey Methods 
and Summary." The effects of VMS user needs upon the VMS 
certification are further discussed in sections: "VMS 
Certification, Suggested Objective Tests" and "VMS 
Certification, Suggested Functional Evaluations." 
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Confrootad wiidi incrMaing airport traffic, riaii^ fttal costs, and now 
aircraft ayutmmM, tha Thk. and airlines have bean expanding the role of 
aircraft siauiation in the training of pilots. Recent advances in 
sisHilation technology, especially in the areas of visual systoas and 
aero aodelling, have enabled the airlines to acooapli^ all of th^ir 
transition ai^ up^ade training objectives using siaulation. Now, a 
pilot training to hold his sane crew status (e.g., co-pilot) on a new 
aircraft systMt (i.e. , transition train) or one training to t^>grade his 
crew status (i.e., upgrade train) c«i receive all of his training, 
including his chedc-ride in sosw cases, in an aircraft sinulator. 

While optiaistic about the training effectiveness of total siatulator 
training program, both t)ie FAA and t!'.e airlines are facing the 
difficult task of evaluating and certifying specific aircraft simulators 
for use in such pacograns. The FAA has published two doctments 
(reference 1 and 2) which describe and coRment u^n the new FAA 
simulator certification process. 

The new FAA simulator certification process eaphSsisas determining and 

i 

evaluating sinulator performance. Nhat is the visual scene content? 
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Mukt is ths visttsl systsoi rsspoiiss Nhst srs ths control 

forces dynMics? Mist srs ths ssro sodsl parssstsrs? Row nsy sxss 
of notion srs sisulstsd? thsss srs typicsl sUsiIstor psrfonuncs 
isssss. Ihs scojps snd nanissr of ths psrfomsncs issoss thst srs 
rsviswsd vary based upon tbs type of training ths sirltns is proposing 
to use ths sisulstor to do. 

Ths Fhh certifiestioa process has been or^mised into ths following 
three categories z 

Phase 1. Phase 1 approval certifies a sinulator for use in the 
current landing nansuver training for transition training. Phase 1 
is designed to encourage upgrading of sisulator eguipatent. 

Phase 2. I^ase 2 certification will provide enhanced training 
capability by expanding the ability of sisnilators to .^rtray more 
realistic visual scene*' and flight characteristics and by iiig>roving 
simulator xeqpcmse dynamics. Phase 2 certification will allow both 
transition and upgrade training. 

I 

Phase 3. Phase 3 approval will allow all but the static aircraft 
training and the line check to be conducted in the simulator. Due 

t 

to the scope of the training and the possible low experience level 
of the training candidates, a high degree of similator fidelity and 
realism is mandatory. 
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THa mterlyinf of tho pctoodint is tiisi incarossss in siaolator 

trsittinf sffsetlvHiMS osn bs had only thxooi^ cogf^ondintf iaeraasas 
in siBttlator fidaiity. It h>s boan coLmo iii^iik tbs aviat^ oonanni^ 
for sianlator training affa|etis«nMS to agnatid with sinnlator 

' * 'll 

I ; I \ ^ 

fidality. Ibis has occa rr ad swan ^oabb data doOs anist (rafaranoa 3) 

’ * 

idiich revaals that substantial wwnnta of trainii^ can ba acoonplishad 
using xalativaly low fidality aircraft sianlatmrs. Ibe assvmption tlmt 
hi^ fidali^ is raqairad fpir hi^ quality training is so strong* 
howavar* that thosa «dM> tesi^a* certify* and use sinnlators adhere very 
closely to fideli^ concepts and criteria in «ihacever they do. 

The tern fidelity is quite vague and overused. Onfortunately* this 
docusnnt will <k> little to refine the term or to refrain from its use. 
Fidelity* as defined by the PAA and airlines* relates to many issues 
almost too ntnmrous to mention. Qnbedded in the term "fidelity* is some 
ccHicept that the cues provided by the simulator are approximate 
representations of those presented to the pilot during actual flight. 

The sore that sinnlator cues duplicate those of the actual aircraft* the 
more "fidelity” the sinulatgr is purported to haye. The scope of the 

I 

definition is alhrsdngly brMd and is shoeing no signs of being refined. 

' I 

TO date no one has been able to decide which of ipie many possible flight 
cues are really relevant to > the training of pilots and %ihich are not. 

Result? In one way or another* any sort of mot on* visual scene* 
instrument* aircraft dynamic* aircraft system* or weather cue has been 
argued persuasively as relevant to creating a high quality aircraft 
simulator for pilot training. The outcome is thAt the utmost "fidelity" 
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1> xxi ^t alMM^ •vuty aap*ct of olreraft slaaloter dosl^n. 

Accordingly^ the FAA ocrtificotlmi j^roecce ottMipts too detoondno *'ho 

t 

"fidolitoy* of oil tohoM vorioos aopoctos of tohe oianlotoor. 

In tho FAA cortolflcotoimi jpcoceos toho fidelity rogoirMonts in«;ease in 
nunber and oeopo as toha training role of the sianlator is expanded. For 
instance* phaiM 1 requires only a three axis notion presentation* 
tdisreas phasg 2 and beyond reqsires tohe eqoivalont of a six axis action 
sinulation. 

! 

The FAA fitelity certification tests and criteria are orgoiized into 
three s^^ate sets ndii^ also vary according to phase. Those sets are: 
general reqoirene'its* objective perfomance tests* and foictional 
evaluations. 

General reqairesnnts include such criteria as the nusber of axes of 
action sinulation* visual scene content* and weather sinulation 
capabilities. Host general requirements are certified by inspection. 

Objective perfomance tests are more sophisticated and engineering 
oriented and are based iqpon objective criteria, such as frequency 
response. Visuad systen* notion system, and control response dynamics 
are tested in this set of tests. One significant and typical 
specification is that the naxinua acceptable visual systen delay is 
300ns (phase 1)* AnoUier is that tohe visual, notion* and instrtanent 
systens shall respond to pi.ot Inputs within 150ns, but not before the 
tine «d)sn the airplane would have responded under the sane conditions 
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(phaM 2 and bayrnid). Bow adaq^ta sianlator pacfoxMBca la cm ba 
raadlly datandnad uaiinr thaaa typa of taata. 

anotbar axaa of mjactiva parfoxaanca taating inwolvaa aaro nodal 
analyaia. Aaro nodal parfosnaaoa la taatad by flying given nanaovars in 
the ainolator. Iba aero nodal output obtained fron the sinulator 
coaq^tar is than oonparad with actual aircraft perfomanca data 
collected pravioualy during airborne test sessions. A paraneter by 
paranatar conpariaon of aero nodal and actual aircraft data trends is 
done subjectively and a^.nulator aaro nodel changes are nade accordingly. 
A special FAA naticmal test taan, cosposad of experienced engineers and 
pilots, assists the airlines in perfoming all c^jective perfonnance 
tests and all general requirenents testing. 

At the outset of the drive to ei^and sinulator use in airline pilot 
training the FAA heavily enphasized objective criteria and tests in 
evaluating and certifying a sixnilator. For instance, at one tine 
changes to the aero nodel based upon pilot opinion data were forbidden 
if any actual aircraft flight data could be found to substantiate the 
aero model paraneters. According to airline officials, the FAA 
encountered numerous problems with user acceptance because of this 
policy. Pilots continually con^lained that a simulator lacked fidelity 
in one aspect or another and should be changed. However, because such a 
change would have caused a loss of simulator certification, the required 
adjustments were not made and the coiq>laints continued. 
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BVMktually« KcMMV»r« ftmetional timlaations gaii^ in i^portanca. Thasa 
tastSf abidi ara dasignad to fnrtliar axplora and avalnata aimlator 
fidality, raly iqpon tha paroaptions and cq^inicms! of aalaet pilots on tha 
PAA national taas and within tha airlina coaaninity. A fnrotimial 
spacificatioh would ba# for axaaipla, aftar roll-«at cmto final approach* 
tha runway should appaar as it would in the actual aircraft at the 
distanca. Tha critaria for cowpliance «dth such a specification is 
coapletaly suibjactivaly defined hy the siaulator test pilot. This is 
unlike objective tests idiara coapliance is more observable. Siaulator 
perfonMnce can be naasured outwardly and coapared with discemable 
criteria when perfoming objective tests. To perform fvmctional 
evaluations* pilots ara required to perform a given set of maneuvers and 
subjectively evaluate simulator performance. Pilot opinion data is then 
aggregated and interpreted and the appropriate changes are made to the 
simulator hardware and software. 

Without the functional evaluation process* there would be no way for the 
FAA to certify that the simulator perfoira perceptually as required. 
Objective performance tests are isolated and insufficient and cannot 
relate directly to the perception issue. They cannot provide sufficient 
cause for accepting a simulator* but can provide a basis for rejecting 
one or a means of directing the use of functional evaluations. For 
instance* oncd the visual system frequency response has been measured 
objectively and accepted* attention can be focused on visual scene 
content* tdiich would be evaluated subjectively using functional 
procedures. 
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i < 

1 

U.S. hlk IKHKX, MWy, AMD NAVY 

i I 

The military haa also baan jj^raasing its use of siaulaticot and has been 
deeply involeied with tevelo^ihg simulator certification procedures. The 

I ' 

certificaticm proeesMs fori the Amy, Mavy, and Air Force are similar to 
those of the |faa. All involve some set of general reguirments and 
objective tests. And, all irely heavily on a pilot-in-the-loop type of 
functional evocation. 

Again, as it was with the FAA certification tests, the military tests 
are aimed at detensining and evaluating simulator fidelity. The 
military also feels that in order to achieve high levels of training in 
a simulator, the simulator must be of exceptionally high fidelity. The 

military defines fidelity even more broadly than do the FAA and the 

i i ' ' 

airlines. T^e military is jidt only concejmed with the fidelity of the 
simulated aircraft, but it is also requirjing that a variet/ of high 

! . ' I 

fidelity ccmibat environments be presented through visual simulation. 

! 

• 1 } 

For instance, fighter ^attack jaircraft sinsilator visual system 

specifications have called for the slnul^^t.ton of enemy missiles and a 

1 

variety of terrain. Although the airlines are Interested in presenting 

a quality visual image of a runway and surrounding terrain, the 

1 

military's rsjquireBients for terrain and scene detail simulation far 
exceed those of the airlines. 
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A* a raaultf tha ■llitary aianilaW spaoitfications oMtaia nuaarout 

i ! 

, i 

functional spaeifloatlcna. For *xaavla« |a flghtar-attaeh aircraft 

sionilator visual systMt spacification might raquira that tha pilot ba 

I 

providad with sufficiant high fidality visual cuss to perform i low 

I 

altitude f high spaed penetration maneuver. Such a specification 
requires a pilot-in-the-loop evaluation tio determine con^llance. No 
objective test exists. To be sure# howevjer# military specification also 
contain objective type specifications, such as visual or mot on system 
response criteria. In either case, «diet)wr functional or ch.''.ctive 
criteria are employed, the ultimate goal is to specify the fidelity of 
the simulator. 

However, at times the militairy hat questioned its reliance upon the 
specification and determination of simulator fidelity as the primary 
indicator of simulator training potential. The Tactical Air Ccmotand 

I 

(TAC) is about to perform a series of tests under the guidance of the 
Air Force Technical Evaluation Center (AF^TEC) to evaluate how well pilot 
evaluations of simulator fidelity correlate with empirical measures of 

simulator training effectiveness. ’ 

I 

Further, the military has also attempted |:o tcike another approach to 
defining simulator performance requiremen|:s other than by simply 
attenptinq to t-pecify the highest fidelity simulator possible. That 
approach, called Instructional Systems Development (ISO) , is based upon 
a breakdown of a aircraft's operational missions into a set of discrete 
pilot tasks. Each task is then analyzed and assigned a set of pilot 
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slUll raquir^BUittf «fhich ace then cevlewled to detendne whet kinds of 

'i I 

tifsining Mdlie would be neeeeaery to develop the required skills. 

’ ’ i 

I 

1 

' i 

j 

The ISO process seens to be e theoretically appropriate approach to the 

I 

' 'I i 

design of sianilators as pilot training devices. The process* however* 

hiLs been plagued by problems. The ISO process requires high level 

: ! 

management support to succeed and that appears to have been lacking. In 
the Air Force* for exasq>le* ISO teams are' appointed within each 
operational command where* due to excessive workload and limited 
expertise* those teams have been unable to contribute much to the 
specification process. Their major task and most significant 
contribution has been to devise the trainjing syllabus and other 
instructional pTOcedures to use in integrating a given simulator into an 
ongoing pilot training program. 

The ISO process is* however* hindered by more fundamental problems. For 

those of you v»to have either performed ori reviewed a task analysis* 

1 

certain thinge be me obvious. Performing a high quality task analysis 

' very* very hard. Often the definition! of tasks is vague and 

! 

arbitrary and, the subsequent skills analysis is c(»apromisec from the 
start. Even 'if the teek /-nalycis were exjict and accurate* deducing 

! I 

which skills are pertinent to the performance of which tasks is also 
quite difficult. It should be of no surprise that determining training 
media specifications using such a set of jpilot skill requirements can 

often result in a product of poor quality'. The ISD process lacks 

! 

support for good reason. TKe process* or any _>rocess like it* can take 
years* cost ntilllons* and lead to a product of questionable quality. 
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only tluRMiqh ti>m um of 019^ fonjteMl and tho! tn c i r oMoa avoiUbUlty 
of iaprovod te^alqisMo and data e«k laprovaoenta in tha quality of I8D 
typo pxoduets ba attaiaad. 

It ia not «»itliimt warrant tbto that tha aviation coaauntty has reliad 
19M1 tha detaminaticm of ainulator fidelity as the priaaury (autd periu^s 
(mly) a priori indicator of a ainulator* s training potential. 

Praviooaly* nilitary and airline pilots were trained socoessfully using 
the aircraft as the prinary training nedia. The extensive use of 
siaulation as a training nedia is recent and the reliance simulator 
fidelity as a certificaticm yardstidc seems warranted in light of the 
success of pre-siaulation pilot training programs. 

The military has not published any documents which discuss and define 
all the phases and procedures involved in how it specifies and certifies 
a siaulator. Further* the Specification and certification process 
within eadi service involves many distinct organizations* so a review of 
the whole process would have to encompass the viewpoints of all the 
participating organisations. 

\ 

However* personnel within certain participating organizations have 
written documents trtiich are relevant (references 4 , 5 , 6* 7 , ard 8). 

One (reference 4 ) discusses how to use flight test procedures in a 
functional evaluation of the fidelity of the simulator aero .'vlel. 
Another article (reference S) discusses the use of pilot-in-the-loop 
analyses and objective tests in the determination of helicopter 
simulator fidelity. A third (reference 6) presents sets of objective 
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criteria foi^ •iaulater visoal iad aoticm systeMt but does not praswit 
ttvidtfiio* or a strong rational# to support the criteria. A fourth 
(rstersae# i) discusses in gonaral terns hov to collect and use pilot 
opinion dat4. koA, a fifth (reference 8) describes a device which can 
nonitor real^tiae sinulator perfomance. 
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Any discussion jof sisnlstor csrtificstiiHi soon loads to an exaoinaticHa of 
the simlator desiga ^pacifications* it is also true that questions about 
the validity of the certification invariably raise questions about the 
validity of the pacifications, whether they be objective or functional. 
And, %diere funetiotial specifications are involved, additimtal questions are 
raised concemikiq lAether or not the functional specificatimis have been 
set. 


In both the nilitary and tte FAA sisRilator specifications have been 
selected to create the highest fidelity sisaxlator possible. Both <^jective 
and functional specifications and tests have been geared toward this goal. 

Meeting a given objective criteria, such as one for visual system frequency 
response, can create a certain satisfaction. But, that satisfaction can be 
short-lived. The workings of human perceptual processes aire quite complex 
and not yet well understood. Too little is known about how a pilot sees, 
hears, and feels to allow simulator design engineers to %o:ite a set of 
objective engineering specifications that will s .isfy human perceptual 
requirements and create a high fidelity simulation. Objective performance 
tests, then, in turn, cannot totally describe the quality of the 
simulation. Functional specifications and evaluations are needed as well. 

However, functional tests and criteria have not resolved the simulator 
certification issue. Functional evaluations rely pon the collection and 
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intarpc«tation i^f i^ilot opinion dnta. Sinco tint quality of pilot opinion 

{ 

data can vary qiraatly as a function of the data collection prbcedures» the 

I 

value of my functional evaluation cttmot be judged until the circonstances 

I 

i 

surrcMinding how! tha data was collected have been described and evaluated. 

. ! 

It is qeestionSble if those who certify sinnlators have given the pilot 

' I 

opiniim data cglloction procedures sufficient atten t i o n. 

i 

i 

, ! 

One reason for idle q^arse attention pilot opinion data collection 

procedures hava^<eoeSped magf he due to the way functional ^ecifications 

1 ; 

are writ^n. authors of functicmal specifications rarely (tefine how 

the pilot opinion data should be collected or «diat kind and aiKMint of 
variability in pilot opinicm data should be tolerated. However, a warning 
should be sounded. Hot any set of pilot opinion data collection procedures 
will do. Great care mst be taken in creating such procedures, for opinion 
data can be easily siishandled. 

Oiffercoit approaches to specification and certification are needed 
depending tqpon how the sisnilator will be used. For exasple, one F-16 
sinulator could be used for pilot training, another one could be employed 
as a research tool for investigating P-16 handling qualities, and a third 
could be used to research the effects of different pilot training methods. 
Even though each of these sisnilators would be an F-16 simulator, they would 
probably perform differently. For exaflq>le, althou^ the basic aerodynamic 
model should be the same for all, certain portions of the model might be 
modeled in greater detail for use in handling qualities research than for 
use in pilot training or pilot training research. Other differences across 
these three simulators %rould probably also exist. In turn, different sets 
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i ‘ 

of •pocifioati^ks and oartification pcocaduraa ahould ba in affact for 

i 

aach. 

• « 

Mow than; ahat doaa tba procading hava to do with certifying a research 
sinnlator like the VNB7 tha role of a research siaulator is a varied one. 
Mot only do the general parforaanda raqaireaants of the research siaulator 

t 

usually differ froa thoda of a training sinulator. but a research 
siinilator*s perfamatica xaqptixananta dbu^ regularly *s new resaandi tasks 
are proposed, ib fao4« tha cdiaaga can ba so great that, for exMple, a 

research siaula|tor configured as an P-16 could be reconfigured as a A-10 

( 

for a new resea^di project. 

The constant change in perfomance requirenents of a research simulator 
such as the VMS will nake its certification an ongoing, evolutionary 
process shaped almost entirely by the demands of the research task. Given 
that the research tasks dictate the performance requiroaents of a research 
Simula' any discussion of simulator certification must first involve an 
examination of the research issues facing the simulator user community. 

Once these issues are defined, one can determine which aspects of the 
simulatimi must ba of the highest fidelity and «ihich aspects are not that 
critical. With this deteradnation made, then more attention can be paid to 
how M certification process should be conducted and what kinds of 
certification procedures, both objective and functional, should be 
performed. 
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Daring th« fbnths of July uid Aagost 1981 a survay was oondacted of 

' t 

potontial Vsirtical Motion Sianiator (VNS) usars within the msA/tass 
oosnonity. kha potontial osari wars MhSh, Mavy or Any affiliatad 
anginaars an|d sciantists «dio wdra involved with STOL* VTOL and/or 
halicoptar toduiology dovalopon^it. All usars ware intarviawed in person 

: j 
1 

and each intiucviow ineludod* hi|t was not linitad to* discussion of the 
following t^ies* 

j ! 

. I 

1. Ganaral progran description and reasons for using sinulationt 

i 

2. 9rogm goals, 

3. Raksons for those goals, 

4. Tasks to be parfoxnad in the sianiator (e.g., landing), 

5. Parfoxswnce neasures, 

6. Visual system requiments (including a discussion of computer 
generated image (CGI) technology) , 

7. Motion requirements (special engrfiasis on discussing the value 
of the vertical motion capabilities of the VMS) , 

8. Potential problem areas, and 

9. Future programs. 


80RVBY NBBiODS AND aNMARf 


After collecting the user comments (reference appendix B for interview 
data) , the eosnents were organised and reviewed and a survey synopsis 
table was prepared (reference Table 1). As sho%m in Table 1, the VMS 
user coMsunity is very hooogeneous, with the predominant research 
interest beihg in handling qualities research. Such interests require 
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Table 1 

Survey Synopsis* 


Program 

Goal 

— 

Aircraft 

■•■■■ 

Task 

Equipment Requirements 

High 

Speed 

Computer 

Wide 

Angle 

Visual 

High 

Scene 

Detail 

Vertical ' 
Motion 

NAVTOLAN 
C. Paulk 

Determine 

display & 
control sys 
design 
requirement 
for landing 
on a ship 

Helicopter 

3 

Landing on 
small ship 

4 

1 

4 

3 

ARMCOP 
L. Corliss 
D. Carico 

Determine 
effects of 
engine and 
fuel systetr 
dynamics on 
handling 
qualities. 

Helicopter 

Nap-of-the 
Earth (NOE) 
flight 

3 

2 

1 

2 

698-VTOL 
B. Lampkin 
S. Wilson 

Determine 

effects of 

vehicle 

design and 

operating 

pix>cedures 

upon 

handling 

qualities. 

VTOL 

Cruise, 
hover , 
transition 

4 

1 

4 

2 

HELI IFR 
R. Forrest 
V. Lebacqz 

Determine 
effects of j 
IFR and VOR 
approach St 
control sys 
design on 
handling 
qualities. 

! Helicopter 

IFR & VOR 
approach 

5 

5 

5 

2 






















Table 1 cent ’ d 
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Equipment Requirements 

Program 

Goal 

Aircraft 

Task 

High 

Speed 

Computer 

Wide 

Angle 

Visual 

High 

Scene 

Detail 

Vertical 

Motion 

RSRA 

J. Jlnkerson 

Determine 
effects of 
new rotor 
design on 
vehicle 
performance 
& handling 
qualities . 

Helicopter 

All typical 

helicopter 

maneuvers 

1 

1 

4 

1 

ADOCS 
£. Aiken 

1 

j 

Determine 
effects of 
new control 
system & 
display 
concepts on 
handling 
qualities & 
pilot 

procedures. 

1 

Helicopter 

NOE flight 

i 

1 

1 

4 

_ _ . _ 

1 

i 

1 

2 

ssv 

R . Bray 

t - 

Validate 
performance 
of specific 
control 
systems on 
the Orbiter 

Space 

Shuttle 

1 

i 

Approach & 
landing 

1 

1 

1 

5 

4 

i 

3 

1 

i NOTAR 

•’ G. Churchil 

i 

i 

t , 

i 

Evaluate 
control lawsi 
8i control 
system aug- 
mentation 
concepts for 
NOTAR air- 
craft . 

NOTAR (no 
tail rotor) 
helicopter 

Low altitude 
low speed 
precision 
control ; e. g: 
hover 

J 4 

2 

4 

i 

1 























Table 1 cont ' d 
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** 


All comments and ratings were based upon the author's interpretation oi user 
comments. 

Rating Scale: 

1. Definitely required 

2. Often required 

3. Most likely required 

4. Maybe required 

5. Will not be required 


o 
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, { 

that th« MMarch si»>lator dyhMdes b« of tho hl^iMt quality. Othar 
rasaai^ ooncams, such as the need to perforn high energy^ low altitude 
saneuwers* will create a need lor a visual systep with good dynanics and 
high detail scene content. These issues and others have been revie%ied 
and the following four areas of concern are proposed for further 
discussion: 

1. CoB^ter systOBS (CDC 7600) perfomance. 

2. Motion systen software perfomance. 

3. C6l systen perfonkance and use. 

4. SiiBulator systen dynasties response characteristics. 

Coiaputer Systen Perforstance 

One user professed a distinct need for a high speed conputer capability* 
while several researchers indicated that their future programs would 
profit from and may even require such cong>uter capability. This, in 
concert with the fanfare surrounding the acquisitions of the CDC 7600, 
may have created high etqieictations. Disappointment, perhaps even 
disillusionnent, would ensue, should the CDC 7600 not perform as 
expected. 

Motion Systems Software Performance 

Nearly all VMS users felt that the motion system vertical travel 
capability was an asset. However, those users whose research tasks 
involved high energy maneuvers, such as nap-of-the-earth (NOE flight) , 
had reservations about the motion systems software. In particular. 
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iOF POOR QUAUIY 

i 

thi«M uMrs !«iqpresi«d doubtc about iriiether or not the t#ash-out schema 
was optimist for NOB flight. 

It was thou^t that facilities engineers sdght have different motion 
system perfbrmance goals than tesehrchers. Facilities personnel, it was 
postulated, may be more orient«|d toward equipment performance 

characteristics and llmitaitionai, while users are more concerned with 

i ] 4- ’ ' 

creating aalimvittE^n^f ^ to performing their research tasks. 

' 'I *5 ^ ’T. , ‘ ; . 

This differ&m in' may be producing aiotion system software 

optimized to^iezplbit hardware capabilities rather than to conplement 
research tasks, such as NOB flight. 

CGI System Perforawmce and Use; 

Concern was raised edx>ut whether or not the present CGI data base would 
allow pilots to perform NOE flight. Specifically, users wondered if the 
visual scene would present sufficient detail to enable pilots to perform 
NOE flight as though they were flying an actual aircraft. Should the 
CGI data base provide insufficient detail, on-line operational chauiges 
to the visual scene would be required in order for the researchers to 
create the desired test environment and to collect data within their 
assigned simulator time. 

The changes to the visual scene that are envisioned are of the "quick 
and dirty" variety. A conplete remake of the data base in order to 
create a scene perfect in every detail is not under consideration. What 
is under consideration is how to change the texture-less scene presented 
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by th« COI so that stt£flcliint oust art avallabls to j^csptually dsflna 
a gcoimd plans at low altltudss. 

For axanplSf whlls ths author was with the Air Force Humam Resources 
Laboratory, researchers atteapted to use the Advanced Slmilator for 

I 

Pilot Training (ASPT) in cpnducting an A-10 low altitude penetration 
study. In that stud^ pilots were required to fly at 200 feet 
above~ground-*lsvel ahd at speeds of 400 knots through hostile enemy 
territoiry. tlnfortunatsly, initial attea^ts at performing this study 
were unsuccessful because pilots could not maintain the 200 foot 
altitude requirement without crashing. The flat, texture-less valley 
floor over %fhich they flew provided no depth or altitude cues. Only 
after the valley floor was peppered with numerous pyramids were pilots 
^d>le to perform the low altitude penetration task. The idea of using 
objects such as pyramids to generate the required cues occurred as a 
result of trial and error. Further, the spacing and size of the 
pyramids %rere determined only after numerous attenqpts. 

NASA researchers, faced with similar tasks and hardware, may have to 
mimic Air Force researchers and resort to creative trial -and-error 
methods to adapt their CGI visual scene to low altitude tasks. Assuming 
that CGI scene modifications will be needed, users questioned how those 
changes would be made and which organization and which personnel would 
make them. 

All users professed a lack of expertise Jn CGI technology, which will 
make them very dependent upon facility personnel to perform and . ort of 
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CGZ so«n« clung*. Should it b* required that users conduct their 
reseaixh within the sllottsd tiae» users and facilities CGZ people will 
probably have to be in close and continuous contact at the start of a 
user's sinulator tiiM. Otherwise* the required CGZ scene changes will 
take overly long to InpleBent and evaluate* leading to substantial 
delays and wasted sinulator tine. 

A second CGZ. issue WAS raised. The CGZ dynasties were of concern to 
almost all rthearchlirl. nieiir rtsearck involved handlir ntities 
evaluation* Which would rely heavily upon visual scene dy. ,u.>..cs. 

Without high quality dynamics* pilots would not be able to properly 
control the aircraft nor accurately rate its handling qualities. 

Simulator System Dynamic Response Characteristics 

As one reads the following it will become obvious that the preceding 
three concerns* computer systems performance* motion system software 
performance* and CGZ system performance and use, can in part be 
considered subsets of the fourth concern* simulator system dynamics. 
Simulator system dynamics, as shown in Figure 1 * is depicted as the time 
required for the simulator subsystems* the visual* motion* instrument* 
and audiotory subsystems to respond to a given pilot input. Actually* 
simulator system dynamics should be described not just in simple time 
lag terms* but also in such terms as bandwidth* ph..se lag* arl amplitude 
ratios. Zn any case* these and other similar measures serve to describe 
a system's dynamics and the dynamics of the VMS are of concern to VMS 
users. 
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The users often qu4sticmed how well the totel sianlator systoe wmild 
perfone dynasdcelly* tdiicb raised such Issues as$ Nould the sisalator 
faithfully reproduce the aircraft dyaaaics as furovided in the aircraft 
■ath aodel? If not* what kinte of dynamic performuice defici«icies 
would be occuring?: Ifhi^ sianlator stdbsysteas (e.g.* visual systems) 
would cxm tribute the most to the simulator system dynamic performance 
deficiencies? Could the performance deficiencies be elisdnated or 

substantially reduced^; If not* idiat wmild be the effect of sisnilator 

! ■ 

performance deficihncilks bn pilot perception and performance and hence 
upon the validity bf handling qualities data? 

Of the four major concerns raised here* the issue of total siimilator 
system dynamics is* as judged by the interviemr* the most critical one. 
In handling qualities research* the simulator ^ the aircraft and the 
performance <^f the simulator is evaluated as though it were the 
aircraft. Should the issiMS of total system ^und subsystem dynamics be 
ignored* the aircraft dynemtics so carefully constructed and calculated 
in the aero model cosq>utations may be seriously distorted by dynamic 
deficiencies of downstream simulator subsystems. The issue of simulator 
dynamics needs attention if the quality of simulator handling qualities 
data is to be assured. 
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VMS CBWmCliTZON 
SUGGESTKD OBJSCTIVB TESTS 

Thcougtaoat tha pvacading discossions tha c«naleon-llk« tem« fidelity, 
has bean a fttndaaantal chhcapt, aithar implicitly or explicitly. Yet, 
its Maning vas dianoitig sid»tly froai discussicut to discussion. To the 
Fhh the tarn hl^ ho^idtittions, to the sdlitary another, and 

to the appears to have had yet a third. 

-\'3r . 4 ^ ' 

The changes in ^ie aaani^ df the tarsi fidelity can be directly traced 

■ i ' 

to the siMilatov user coiiainity and to the role it assigned 
sisiulation. Throu^ the intaracticm of the user and the role 
simulator fidelity requireemnts took shape, and, in sonm user 
environments, continued to evolve and change to suit changing 
user-role relationships. A heterogeneous user commute ity and/or a 
conple^, multi -dimensional role for simulation could make the meaning 
of fidelity alarmingly broad and the task of providing an adequate 
simulation environsient terribly difficult. Or, at the other end oi 
the spectrum, a homogeneous user and a narrowly defined role could 
refine the meauting of fidelity and make creating, evaluating, and 
certifying the simulation easier (but not necessarily easy) . 

As discussed, the VMS user community is homogeneous and is 
concentrating on simulating and evaluating the handling qualities of 
some type of ciqierimental aircraft. Naturally, to them the dynamic 
fidelity of the total sisnilator system is crucial. Without dynamic 
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fidslity* tlM validity of their research data could be challenged 
easily. 

Wiat afpears to be neitded as a first step toward certifying the is 
to develop an objective# engineering oriented technique for neasuring 
the dynamics of the total simulator system. That technique should 
ei^Aiasize measuring the responses of the visual and motion systems# 
since those systems provide the bulk of the relevant cues and are 
probably the source of mMt of the syst«a dynamic deficiencies. 

Neasurmsents taken at the output of the simulation cosputer would be 
insufficient because they ignore the dynamic effects (and possible 
distortion) introduced by downstream simulator subsystois. 

Measurements :.Z the dynamics of a given piece of subsystem hardware, 
such as the motion hardware# tiould also be insufficient because the 
dynamic effects of the software that controls that hardware would be 
absent, as %rauld be the effects of that software-hardware interaction 
upon the total simulator system dynamics. 

However# it is also realized that measuring the total simulator system 
response may be difficult and lead to ambiguous results. For example# 
non-linearities in the aero model may corrupt attenpts to measure 
total system dynamics. Should that be eiqpected to occur often# it may 
be more productive to concentrate upon measuring and modelling the 
visual and motion subeystems and include in such models all relevant 
issues that affect total system dynamics. 
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In any caMt %diatlMr ifcotal systaa or just svtbaystMi dynaoics are being 
eraatned, it is crucial that the subsystea response, such as the 
■oveaent of the cab itself or the visual scene, be accurately 
aaasured. The technology for Measuring the sisulator cab notion is 
available. Through tiie use of the appropriate acceleroaeters and 
Boticm systea position sensors, the responses of the notion system 
could be accurately aeasured. 

The tedinology for ashsuring the aoviaent of iaages on a CRT scteen is 
less developed, however. Mhat would apparently be needed would be a 
aeans of converting li^t level dianges into voltage changes. %rhich 
then could be reduced to nuaerical trends. A possible technique might 
eitploy a stylised visual image such as a white/black bar graph and 
panel of sensitive photoelectric cells that would be strapped on to 
the CRT optics and connected to a cosq>uter system. A given pilot 
control input, when played through the simulator system %iould cause 
the bar graph to move accordingly. The bar graph movement would, 
hopefully, cause changes in the photoelectric cell voltage output. 

The fluctuation in voltage output could then be organized and 
interpreted by the computer system and Its software. 

Other means of measuring visual response could also be used. Matching 
visual scene response with that of a preprogrammed superimposed visual 
image has also been suggested. In any case, the issue is to determine 
and measure the dynamics of the visual system 2tnd relate that to the 
dynamic perfonumce of the whole simulator system. Any objective 
accurate, reliable, easy->to*use technique will do. 
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EaM-of-uae is quits iaportant. Giusn tlis varied role the VNS will be 
required -to play (reference Table 1) * certifying the VMS will be an 
on-going issue. Each new reaear^ task will chai^fs VMS perfomance 
and require that sianilator perforaanoe be re-validated. Thus, any 
certification technique will be used regularly and often and atust, 
therefore, be easy to use. 

Other issues warrant discussion as well. NASA has encountered 
difficulties in adapting the CDC 7600 to perfom real-tine sianlation. 
impropriate personnel have been assigned to deal with these 
difficulties and their progress is being nonitored by NASA nanagenent. 
Once the systen software is developed and approved, the dynamic 
performance of the simulation coaputer (reference Figxure 1) should be 
documented and re-examined regularly, especially after system software 
changes or large, conplex aero sodels have been Introduced. 

Measurement of simulation c<»aputer dynamics could be accomplished 
using a variety of hardware based objective techniques. 

Besides the dynamic performance characteristics of the motion 
subsystem, certain more qualitative concerns exist. Those surround 
the issue of adapting the motion software to the peculiarity of a 
task, rather than to equipment nuances. Although motion system 
frequency bandwidth definitely affects how well the motion cues of a 
given task can be simulated, bandwidth is not the only relevant issue. 
Users need to know more about how motion system cues differ 
qualitatively from those commanded by a valid aero model. For 
instance, in a pure pitch maneuver, does the motion system introduce 
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any lateral or roll acoalarations? Su^ issiies could ba avaluated by 
cooparing two tiae hlatoriast tee describing cab acceleratites and 
another defining the accelerations coosanded by a valid aero nodel. 

I 

Even tbmagfi the coeparison would involve subjective evaluation, the 
two observable tiae histories would provide the basis for conparison 
and lend considerable objectivity to the evaluation. 

As an asite, one cause for the ctecern over action subsystem 
performance is a lack of rapport between users and facilities 
engineers. Heither has had time to acre throughly understand the 
other's area. 

Perhaps a useful tool to use to bridge the gap between users and 
facilities engineers would be a short VMS motion system seminar. A 
small group of facilities engineers would be made responsible for 
presenting an in depth overview of the VMS motion system software and 
hardware characteristics at the seaiinar. The engineers could 
complement that presentation with a discussion of how to develop and 
inplement motion curing in a real-time simulation. The seminar might 
also include a presentation by interested users on the motion 
characteristics of motion sensitive tasks. The goal of the seminar 
would be to determine how to adapt the VMS motion system to high 
energy motion sensitive tasks like NOE flight. 

Zn essence, the creation and use of objective tests such as those 
described above would solve numerous problems associated with 
certifying the VMS. Re.nember, however, these techniques will not 
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resolve the %AtoIe certification issae. Other issues do still require 
attention and other types of tests will be relevant. It is only 
because of the heavy enphasis placed on dynamic fidelity by the user 
comunity that objective tests such as those described could solve 
such a large portion of the prc^lems surrounding VMS certification. 
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VMS C^IFICATION 
SUGGESTED FUNCTIONAL EVAUJATIONS 


tOnce objective ! tests such as those suggested previously are developed 
and used, data that defines VMS system dynamics should be available. 
That data will ' most likely reveal that the VMS lacks complete dynamic 
fidelity and t^t the fidelity deficiencies are due to siibsystem 4 
dynamic performance limitations » What will also probably be 
determined is that some VMS dynamic performance deficiencies arc 
irrepairable, leaving the following issue to be resolved: What 

effects do VMS dynamic deficiencies have upon pilot perception and 
performance? 

Answering such a question is important if the value of handling 
qualities data is to be determined. To answer the question will 
require the collection and interpretation of pilot opinion and pilot 
performance data. Deciding how to co^ect that kind of data and how 
to interpret it can cause one to encounter many complex and subtle 
issues. 

One issue is under what conditions can a pilot evaluate the phenomena 
in question. What kinds of circumstances or perceptual inadequacies 
are licd^le to undermine his ability to consistently render a valid 
judgement? How can those things be controlled so that the validity 
and reliability of the pilot's judgement is preserved? (Validity is 
the ability of a subject to focus upon the pertinent Phenomena end 
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accurately evaluate them. Reliability is the ability of a subject 
consistently give the saaie responste to the same situation.) 





For instance, the typical functional evaluation might ask a pilot to 
evaluate simulator I'oll response and determine whether it is 

I 

, ; t 

representative of the roll response of the aircratt. In rendering the 
evaluation a pilot must compare the simulator roll response with his ; 

memory of the aircraft roll respoiiiie. The evaluation process is 

j ^ ji ' 

iterative and Ka^hazard. ' Rarely are the pilots con^letely ! 

I ■ ■ ■ . • ' 

‘ ‘ ' * 5 t i 4 ■ ' 

satisfied initially#! so numerous adjustments to simulator roll 
response are made and evaluated. .The process continues until some 
sort of consensus is reached. 


Usually insufficient attention is given to organizing the sequence of 
changes or to evaluating whether the pilots wort clian^ting thcii 
evaluation methods or criteria during the evaluation. Dovi.sirv: 
pertinent performance measures and a valid moans to interpret 
resolve differences in opinion across pilots ate difiicult proMrms al:; 
awaiting attention and resolution. The basic problem with this mode 
of evaluation is not that pilot opinion was solicited, but hov; it was 
solicited. This mode of evaluation often employs faulty procodnro.s 

! 

that cause data reliability and validity problems; . 

I 

Data validity and reliability problems plague the use of pilot ; 

performance data, as well. Fov oxampio, if a pi I"! can ac u tm tf 1 \ 
glide path using a dynamically deficient simulation. '■viKit doi .- I hat 
mean? Also, what does an accurate track look like? iu>w tjvic’-'ur >1:0 is 


track a 


, 3'1 



it? How< mafiyaccurat«* profiles nust a pilot be able to fly in order 
for a researcher to certify that simulation? Further, must all pilots 
perform to the same level? If not, how many should in order for s 
meaningful evaluation to be made? 

e 

Despite the soundest data collection procedures and the most 
insightful eiqperiiMntal design, one aspect of human opinion and 
performance is never altered, niat is change. From trial to trial or 
from person to person, even though the situation is outwardly the 
same, opinion or performance data will almost always vary, sometimes 
subtly, sometinms drastically. Accurately interpreting that data 
veuriance is one of the most fundamental issues in behavioral research. 
Sometimes variance indicates that reliability and validity problems 
and sometimes it does not. Using a valid means of organizing data 
variance for further examination will help to make data more 
meaningful and to reduce data reliability and validity problems. 


The discipline of inferential statistics was developed to help 
researchers organize and accurately interpret data variance. 
Inferential statistics tests are designed to determine when reliable 
differences in performance or opinion has occurred across a set of 
alternate situations. How the alternate situations are structured and 
presented and how performance is measured in each are crucial to 
determining if inferential statistics tests can be used effectively. 
Often, functional evaluations are conducted with such little attention 
to data collection procedures and performance measures that 
inferential statistics tests camnot be used meemingfully. 


OK fOOB 
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Through the use of appropriate eqplrical procedures » a researcher 

i 

could eaploy statistical tests. By using in^roved en^irical 
procedures and statistical tests, a researcher could make a more 
informed evaluation of the VMS. Since the main issue is to determine 
how VMS dynastic deficiencies may affect pilot perception and 
performance, any functional evaluation must deal with a range of 
simulator dynamics and atten^t to measure how sensitive pilots are to 
changes in simulator dynamics. 

This type of evaluation, sometimes called a sensitivity analysis, 
might proceed as follows: As first step, on an apriori basis select a 

set of frequency bandwidths that would be expected to affect the 
pilot's perception of simulator dynamic fidelity. For exanple, the 
appriori rational may be that pilots would perceive narrow bandwidth 
systems as low in dynamic fidelity ^md wide bandwidth systems as high. 
Next, prograun the visual and motion subsystems to represent each 

member of the set euid expand the paradigm with another variable, aero 

,* 

model stability. Select and include a variety of aero models ranging 
from unstable to st^d}le. Then, create a series of tasks and select a 
group of pilots. Finally, collect the appropriate opinion and 
performance data, including dynamic fidelity ratings and handling 
qualities evaluations, using acceptable empirical procedures. 

By using a more conceptually organized approach and more acceptable 
empirical procedures, this type of sensitivity analysis should provide 
much more information than would a typical functional evaluation. One 
virtue of this approach is that data comparisons can be done 
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statistically and ars now more meaningful. Comparison of the data 
within a given condition and across conditions should suggest how 
reliably the pilots responded and how sensitive they were to a change 
in conditions. Changes in pilot opinion and performance across 
conditions should suggest, specifically, how dynamic deficiencies 
affected the perception of dynamic fidelity and the rating of handling 
qualities. Thus, a researcher would know more about how much dynamic 

I 

fidelity could be 'attained ami how degradations in dynamic fidelity 
could affect handling qualities evaluations. 

Actually, the preceding example needs further development before 
meaningful sensitivity analysis could be performed. But, the concept 
is valid. This type of analysis could provide a researcher with a 
much richer context in which to evaluate the quality of a simulation. 
And, although the analysis is not a complete cure for all data 
validity and reliability problems, the richness of the empirical 
context should provide a researcher with more cues about the quality 
of his data. 

To further develop the preceding, a number of issues must be explored 
in more detail. For instance, regarding pilot selection, what kind of 
pilots should be select 1 and how many? The selection of pilots has a 
great deal of influence upon the kind, quality, and reliability of 
opinion data. Further, if performance data is to be collected, what 
tasks are to be i>erformed and how will perfornumce be measured? For 
collecting pilot opinion data, how will the questions asked and 
what type of responses will be appropriate? Will the questions be 
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opm «nded or stn^ctured with a rating scale? And, if a rating scale 
isi appropriate, irtiat kind of scale? Should the scale be 
unidime' onal or multi-dinensional? Should the scale be a simple bar 
graph or a well defined interval scale? Finally, will the 

’ i 

questionnaire data be classified as ordinal, interval, or ratio? The 
scaling of data has a great influence upon the type of statistical 
tests that can be i applied. Generally, interval and ratio data c£ui be 
used in a larger dumber of statistical tests than can ordinal data. 

I 

I 

The overall exper^Usental paradiagm is isq^rtant also. Should the 
pilots be organizfd into independent groups, each receiving only one 
research conditiod, or should all pilots be ei^sed to all conditions 
(i.e., repeated measures)? Further, if all pilots receive all 
conditions, should the sequence of conditions be counterbalanced or 
randomized? Within a given resf trch condition, how should the effects 
of repeated trials be interpreted and how will the issues of 
accommodation to the simulator be handled? (Accomnodation is a 
phenomena in which experience with the simulation masks memoi'ies of 
aircraft experience, making it very difficult for a pilot to validly 
interpret, evaluate, and use simulator cues.) Further, what range of 
bandwidths and aero models should be selected emd how many intervals 
within each range should be tested? Also, are other kinds of 
parameters besides bandwidth relevant to dynamic fidelity and if so, 
which ones and how should they be tested? These questions are just a 
subset of those that may prove to be relevant, so other questions may 
need to be asked and answered before a valid sensitivity analysis can 
be conducted. 
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A Mnsitlvity analysis could also be dona to avaluilba CGI soana 

) 

flidallty. Using apriorl assun^tlons about what variablas affact scans 
quality, altamata sitv* tlons could be esplorad. l%a outcona of i 
scans functional evaluation , as with many kinds of functional 
evaluations, is particularly sensitive to the pilot tasks used during 
the evaluatior . For one thing, that means that the answers to nu>ny of 
the questions in the two preceding paragraphs may vary from one task 
tej another. For another, it means that the applicability of the 
results will be limited. 

For instance, if the task were hover, one set of questions and 
performance measures would be relevant and if the task were apprc> • 
and landing, another set would be appropriate. The results from the 
hover tests would be most relevant only to similar kinds of tasks and 
the same would be true of the results from the approach and l^-.ding 
tests. It is Important, then, that any functional evaluation of the 
CGI scene incorporate the pilot tasks of interest to thn VMG user 
community (reference Table 1) . Further, scene functional evaluations 
may be aircraft sensitive as well. Fortunately, since the typical VMS 
user is simulating some type of helicopter (reference Table 1) , scene 
functional evaluations should relate primarily to helicopter aircraft. 
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PERSONS CONTACTED 


Lt Col R. MacArgel 
TAC 

Eglin AFB, PI. 

Maj Robert Whelton 
APTEC 

Kirkland APB, N.M. 

Lt Col John Rizinski 
APTEC 

Kirkland APB. N.M. 

Lt Col R. Rogers 
SAC 

Offutt APB, Ne 

Lt Col J. llueller 
APTEC 

Kirkland APB. N.M. 

Lt Col R. Hartog 
MAC 

Scott APB. 111. 

Lt Col A. Meacham 
ATC 

Randolph APB, Tx. 

Capt Dalros 
ATC 

Randolph APB, Tx. 

Dr. Ken Boff 
AMRL 

Wright-Pat terson APB, Oh 

Mr. Ken Potempa 
APHRL 

Brooks APB , Tx . 

Dr . Thomas Gray 
APHRL 

Wilx..ains APB, Az. 

Dr. Thomas Longridge 
APHRL 

Williams APB, Az. 


Mr. Ronald Ewart 
ASD 

Wright-Patterson APB, Oh. 

Hr . George Dickison 
ASD 

Wright-Patterson APB, Oh. 

Mr. James Bassinger 
ASD 

Wright-Patterson APB, Ob. 

Mr. Thomas Galloway 
NTEC 

Orlando, Pla. 

Mr. James Burns 
NTEC 

Orlando, Pla. 

Mr. Walter Chambers 
VTRS facility 
Orlando, Pla. 

Dr. Ronald Hofer 
PM TRADE 
Orlando, Fla. 

Col Deel 

Aeromechanics Laboratory 
Moffett Field, Ca. 

Mr. David Key 
Aeromechanics Laboratory 
Moffett Field, Ca. 

Mr. Richard Dunn 
Aeromechanics Laboratory 
Moffett Field, Ca. 

Mr. Gary McCullough 
United Airlines 
Denver, Cb. 

Mr. Charles Huettner 
FAA 

Washington, D.C. 
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PERSONS COHTACTEP-Cont timed 


Mr. Edward Fell 
FAA 

Washiagton, D.C. 

Mr, Robert Trainer 
CSC 

Moffett Field, Ca. 

Mr. Valter Boeck 
CSC 

Moffett Field, Ca. 

Mr. A. M. Cook 
NASA Ames 
Moffett Field. Ca. 

Mr. Donald Dust 
NASA Ames 
Moffett Field, Ca. 

Mr. David Brocker 
NASA Ames 
Moffett Field, Ca. 

Mr. Herbert Hoy 
NASA Ames 
Moffett Field, Ca. 

Mr. Richard Bray 
NASA Ames 
Moffett Field, Ca. 

Mr. William Cleveland 

NASA Ames 

Moffett Field, Ca. 

Mr. David Key 
Aeromechanics Laboratory 
Moffett Field, Ca. 

Dr . John Lauber 
NASA Ames 
Moffett Field, Ca. 

Dr. Richard Dunn 

Army Research and Technology Laboratory 
Moffett Field, Ca. 

Dr. H. C. Foushee 
NASA Ames 
Moffett Field, Ca. 
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SCHEDULED VMS USERS: 


Jul 81-Dec 82 



User 

Program 

Interview Dates 

1. 

Clyde Paulk, FSN 

NAVTOLAND 

6 

Jul 

81 




18 

Aug 

81 

2. 

Lloyd Corliss, FSD 

ARMCOP 

17 

Jul 

81 




19 

Aug 

81 

3. 

Bertram Lampkin, FHS 

698 VTOL 

22 

Jun 

81 


Sam Wilson, FHS 


19 

Aug 

81 

4. 

Raymond Forrest, T 
Vic Lebacqz, FSD 

HELI IFR 

23 

Jul 

81 

5. 

John Jinkerson, FHI 

RSRA 

28 

Jul 

81 

6. 

Edward Aiken , FSDC 

ADOCS 

18 

Aug 

81 

7, 

Richard Bray, FSD 

SSV 

1 

Sep 

81 

8. 

Gary Churchill, FHTC 

NOTAR 

3 

Sep 

81 
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REASONS 


1. C. Paulk 

- Interchangeable cab fits VMS 

- VMS designed for VTOL concepts (motion travel) 

- Needs wide angle visual to simulate hover 

- CDC 7600 capabilities play a minor role (for now) 

2. L. Corliss 

“ Evaluate value of vertical motion on VMS; Army urges use 
of VMS. Corliss cooperated. 

- Need for high speed computational capability. 50 msec cycle 
time too slow - marginal. Aiming for 25 msec frame time. 

High cycle (i.e. 50 msec) may corrupt value of data. 

- Future need for wide angle visual anticipated. 

3. B. Lamplin, S. Wilson 

698 - VTOL/STOL aircraft program; tilting nacelles design 
by Grumman. 

Thought simulation would be a good idea to use in evaluating 
advanced modeling work now under contract to Grumman. 

High speed computational capability will not be needed until 
testing of STOL/VTOL landing (and landing gear) is performed. 
High speed computer will also be needed eventually when 
high fidelity engine model is available. 

4. R. Forrest, V. Lebacqz 

- Helicopter handling qualities during IFU flights; primarily 
approach . 

Chose VMS because of frequency and travel characteristic.s 
of the motion system. 
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Jlnkerson 


Rotor Systems Research Aircraft test advanced rotors. 

Test handling qualities of advanced rotors that will be 
on RSRA. 

Develop and test flight techniques for advanced rotors. 

Do failure inodes and effects analysis. 

Validate stability augmentation system. 

Evaluate flight computer system, advanced control algorithms 
and advanced control systems. 

Need high speed computational capability for multi-element 
rotor blade simulation. Multi-element blade model neces- 
sary; Standard Bailey models based upon actual aircraft data. 
Since rotor concepts tested here have not yet been developed 
into hardware form, multi-element blade model is only (best) 
way to translate wind tunnel aero data into a usable math 
model . 

Aiken 

Advanced Digital Optical Control System. 

Evaluate controller and display configurations from a han- 
dling quality and human factor (i.e., workload perspective). 
Chose VMS because of motion and wide angle vision capabil- 
ities (4 axis controller). 

Bray 

JSC limited in terms of engineering simulation. JSC em- 
phasizes training simulation. 

Combination of visual and motion capabilitie.s at Amos VMS 
thought appropriate for Orbiter engineering simulation 
research. 
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Churchill 

Frequency response of VMS simulator systems, including 
mot ion > 

rtot much initial interest in CDC 7600 high speed compu- 
tational capability. 

NOTAR (no tail rotor) concept evaluation dependent upon 
dynamic qualities of the simulation. 
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GOALS 


Paulk 

Develop technologies of flight control and display system 
(landing guidance systems) necessary to land a VTOL air- 
craft on board a ship (not necessary to impact a given 
vehicle design). 

Corliss 

Impact design of future h*»licopters . 

Emphasis on control system design, especially engine 
control systems. 

Primarily for high maneuver tasks (NOE). 

Determine outer limits for frequency response, damping, 
time delays, available torque. 

Lampkin, S. Wilson 

Impact design of VTOL aircraft. 

Demonstrate the usefulness of the aircraft configuration; 
especially the use of vanes to control aircraft moments 
at low speed. 

To examine flying characteristics and handling qualities 
of aircraft concept. 

Forrest, V, Lebacqz 

Establish control system boundaries to use in the design 
of safe control systems. 

Determine flight control and flight instrximent requirements 
for optimum or near optimum IFR approach performance. 

Need to determine level of control system augmentation 
required . 
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5. J. JinkersoD 

> Impact rotor design technology 

- Evaluate new rotor concepts 

- Create rotors that are: 

Quiet 

Powerful 

Vibration limited 

- Simulation used to weed out unpromising rotor concepts 

6. £. Aiken 

- Evaluate and demonstrate "flight by light" controller con- 
cept including new and unique controller concept — 4 axis. 

- Develop controller and control laws optimized for aircraft 
(Black Hawk). 

- Examination of a given control configuration just as impor- 
tant as the selection of a certain concept for optimization. 
Impact vehicle design. 

- Need to compare and determine control requirements for day 
vs night NOE flight. 

7 . R . Bray 

- Validate specific control systems configuration; e.g. , 
verify auto land system capabilities in a variety of con- 
ditions: wind, nav. system failure, etc. 

- Evaluate shuttle display concepts, especially with regard 
to HUD technology. 

8. G. Churchill 

- Investigate flight test anomalies associated with NOTAR 
(no tail rotor) concept. 

- Evaluate control laws and control augmentation systems. 

- Develop technology to impact tail rotor design on actual 
vehicle. 
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REASON FOR GOALS 


1. C. Paulk 

- Get miaimun down tb soft zero — zero condition. 

- NASA 

Facilities 

Expertise 

- "Fix the fleet" vs. radical change in fleet aviation concept. 

- Long term Navy interest in landing on board small ship. 

2. L. Corliss 

- Army mission; need to perform maneuvers quickly (time/clis- 
tance) in order to avoid detection and to effectively use 
terrain to mask radar signature. 

3. B. Lampkin/A. Wilson 

Future Navy mission will involve STOL/VTOL aircraft. 

- Need to evaluate contractor progress. Grumman boasting 
loudly. Need to review - evaluate. 

4. G. Forrest /V. Lebacqz 

Use of helicopters increasing. 

- More IFR flight will occur. 

There may be changes in helicopter design that will require 
dramatic changes in control system design. 

5. J. Jinkerson 

Reason for goals is to keep U.S.A. helicopter manufacturers 
on the forefront of technology. 

No new rotor concepts in over 20 years. 
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6. £. Aiken 

- Army mission 

Flight~by“ light less susceptible to electro-magnetic 
influence than fly-by-wire. 

- Other benefits of ABOCS: 

redtiiced workloa^l (?) 

improved flying quality for NOE (better vision) 

7. R. Bray 

- Assure ! appro|>riatehe8s of vihlcle design. 

- Increase confidence in use of equipment. 

- Impact vehicle design, but only when system performance 
probl«ns are discovered. 

8. G. Churchill 

- Tail rotor very vulnerable 

military operations 
excessive flare 

- Other problems: 

hard to maintain 
noisy 

- Elimination of tail rotor would be beneficial provided no 
degradation in vehicle handling qualities occurred. 

- DARPA funded program. 


^^RQUAUTy 
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TASKS 


1. C. Paulk 

- Land VTOL or helicopter on the back of a small ship in 
a variety of weather conditions. 

2. L. Corlisss 

- NOE flight (and other high maneuver tasks, e.g. pop-up, 
high speed stop tO| a hover) 

3. B. Lampkin/S. Wilson ; 

t 

- Cruise: moderate i^ltitude and airspeed 

- Transition corridor 

Cruise- low speed flight 

Examine thruse control and aircraft stability 

- Hover 

- Translation in low speed flight 

Fore /aft 

Lateral 

Vertical 

- Simulate system failure 
Simulate engine failure 

VTOL approach (landing in the future) 

4. K. Forrest /V. Lebacqz 

- Non-precision VOR approach (IFR) 

- Presicison ILS approach using 6 glide path (IFR) 


5. 


J. Jinkerson 

Any/all types of tasks 
Task 
Landing 
Takeoff 
I >»ver 
Noe 


Emphasis 

Heavy 

Heavy 

Moderate 

Light 
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Task 


Emphasis 

Cruise Moderate 

Lateral movement Moderate 

Pop-up Moderate 

Yaw Moderate 

Note, this research is not a task oriented program. 

A typical command to pilot would be, "Exercise the 
pitch axis." 

Searching for control stability and authority problems. 
Aiken 

NOF flight, both Day and Night 
Bray 

Approach and landing 
Churchill 

Low altitude task, low speed 
Hover 
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PERFORMANCE MEASURES 


C. Paulk 

- Measure A. C. performance: altitude rate, roll rate, etc. 

- Measure pilot performance: stick movement 

- Pilot comments 

- Cooper /Harper ratings 

- Intearation with optimal pilot model 

- AwWe must be integrated and evaluated using expert 
opinion 

L. Corless 

« lime between bob-ups 

- Error off intended flight bath 

- Cooper /Harper ratings 

- Attempted to correlate pilot ratings with error scores 
(a problem) 

B. Lampkin/S. Wilson 

System performance measures (e.g. roll rate) 

Pilot opinion 
Navy pilots 
Grumman pilots 
NASA 

R. Forrest /V. Lebacqz 

Need to measure events surrounding basic longitiudinal 
stability 

- Mostly pilot opinion data 
J. Jinkerson 

Primary data source; pilot opinion 
Open ended 

Little use of structured questionnaires 



Aiken 

System performance 

Error scores (flight pattern error) 

Time required to perform a task 
Distance required 
Pilot opinion 

Test pilot- engineering evaluation 
Line pilot- human factors evaluation 
Workload measure 
Pilot rating 

Use of secondary tasks such as the Steinberg tasks 
Bray 

Pilot opinion 

Churchill 

Strip chart data 

Pilot opinion data 

Need to corroborate pilot opinion data with enginooring 
data in order to ascertain if the pilot is responding 
to a characteristic of the aircraft or to some anomaly 
introduced by the simulation. 



VALUE OF CGI 


1. C. Paulk 

- Require wide field of view visual system 

- High detail may not be required 

- Simulation of ship dynamics critical, enen though 
level of detail requirements is low 

2. L. Corliss 

- Not planning to use CGI presently (will let his 
colleagues find out how to use CGI) 

- In future wide angle capability will be required 

- Level of detail in CGI scene content may be in- 
adequate. Need scene as testured as it would be 
for NOE flight. 

Perhaps a gybrid system 

Camera/model- forward field 
CGI- peripheral 

May need to change CGI data base frequently 

3. B. Lampkin/S. Wilson 

Definitely require wide angle system 

- Need high detail for ship environment, depth perception 

- VTOL, lower right- high detail required 

System dynamics critical, expecially visual /mot ion 
interaction. Would sacrifice motion dynamics for better 
visual 

Hover and transition maneuvers especially sensitive to 
visual and motion dynamics. 

- Has used HUD to supplement wide angle vi.sual. 

4. R. Forrest /V. Lebacq?- 

Irrelevant to IFR flight 

- May use, though, for simulating now electronic 
instrument displays 
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5. 


J. Jinkerson 


- Quality of the scene will determine quality of the data 

- Need wide angle visual 

- Low need for high detail (?) 

- Critical vision sensitive tasks (all from precision hover) 

Hover 

Backward 

Lateral 

Forward 

6. E. Aiken 

- Wide field of view required for daytime 

- NOE flight should be a problem. Lack of scene detail. 
Lacks knowledge of how to modify data base. 

Needs to create an environment in which the pilot will 
exhibit "appropriate" behavior (i.e. behave as though 
he were performaing that task in actual aircraft). 

7. R. Bray 

Wide angle visual of little relevance 
System dynamics critical 

- Detail requirements uncertain. Model board display 
used previously with success. 

Expensive CGI may not buy much for landing maneuver 
performance. 

8. G. Churchill 

Present dynamics of camera model board system can hv 
determined and satisfactorily modified. 

Camera model board f icld-of-viow a problem that can be 
tolerated, usually. 

CGI wide field of view would bo an advantage 
Scene content? 

CGI dynamics critical to research goals. 
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VALUE OF VMS MOTION 


1. C. Paulk 

- Enhance value of experimental situation, especially flight 
control work and handling quality. 

- Frugal with the use of motion, use only when needed; value 
of interchangeable cab. 

- Favors 1-1 motion (no washout) 

vertical 

longitudinal 

2. L. Corl’ss 

- Not sure 

Motion washout a critical problem (past experience). 

Large travel capability of VMS may allow 1-1 potion soft- 
ware. Use of 1-1 (i.e. , the elimination of washout) 
would be very helpful. 

3. B. Lampkin/S. Wilson 

Intuitively, motion appears to be a sti’ong cue. 

- Use of motion should aid in acquiring better data. 

4. G. Forrest /V. Lebacqz 

Motion an asset when evaluating minimally safe control 
systems. 

- Vertical travel capabilities an asset, given emphasis on 
longitudinal and vertical stability/control. 

5. J. Jinkerson 

- Tasks sensitive to vertical motion and operation of the 
collective. 

- Pilot needs to perceive and use vertical motion to operate 
vehicle. 
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Sure data will be of better quality due to enchanced VMS 
motion capability (esp. vertical motion). 

Aiken 

Que^'tions value of motion. Reports comments like; fixed 
base/single window and HUD system "better" than VUS/single 
window — no HUD. 

Motion system software may be a problem. 

Need to optimize software for NOE flight (or any task). 
Different points of view: 

1. Researcher: task; cues to pilot 

2. Facilities; har'^ware characteristics, software potential 

3. Result: different motion software would result. 

Bray 

Helped a great deal. VMS travel capabilities helped espec- 
ially in PIO work. Orbiter has unusual pitch and Z axis 
characteristics that can induce PIO. VMS motion helped 
pilots gain confidence about their ability to pilot the 
Orbiter. 

VMS motion also valuable for turbulence simulation. Lower 
frequency simulation capability of VMS provided better tur- 
bulence cues than FSAA type motion. 

Overall higher confidence in the data. 

Churchill 

Vertical travel and frequency band width of VMS helpful. 
(Good data has been obtained u.sing FSAA, however-.) 

Need to model motion system (software and hardware) trans- 
fer function. Need simulator cab acceleration data. Must 
be able to modify motion system dynamics to fit the needs 
of the aircraft aero model in order to assure user of high 
fidelity simulation. 


59 


ORIQINM.PMEIt 
OKPOOR QUALITY 



FUTURE USES 


1. C. Paulk 

- Navtoland program should continue 

- Evolveroent will occur, especially when the specification 
and procurement process begins to check out proposed hard- 
ware and specifications. May not use VMS, use simulation 
for control law development and solving real operational 
problems . 

2. L. Corliss 

- Will be much like the present. (Effects of engine dynamics 
or handling qualities.) 

3. B. Lampkin/S. Wilson 

Research of this sort should continue. VTOL work an essen- 
tial part of the long term Navy mission. 

Navy specifically requested the use of simulation facilities 
for V1X>L. 

4. G. Forrest /V. Lebacqz 

"slung load” simulation 
Autorotation 

advanced rotor models 
ground effects 

5. J. Jinkerson 

- Much like the past 

6. E. Aiken 

- ADOCS will continue for a while 


GO 



7. R. Bray 

> Other S8V sloralatlons planned 

- Other research topics in simulation technique planned as 
well. 

8. G. Churchill 

- Limited flight test and/or simulation test results may 
limit future funding. 
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PROBLEU AREAS 


Paulk 

Will equipment integration occur in time on VMS system? 
Coping with CGI may be a problem. Is the scene appropriate? 
How can we change it? How can the sim. task be made more 
realistic? Can the scene be changed? Is organization pre- 
pared to do so? (Should CSC be more involved? Is staff 
being created to do so?) 

Anyone involved in examining CGI per se? (No) 

Corliss 

Motion washout. Use bob-up as a test task; washout vs. no 
washout . 

Post flight data processing a problem. Tape conversion a 
problem. Could tape format be made compatible: RUNDUM 

tapes don’t play on in-house computer. 

CGI problem: detail and data base modification. 

Cycle time a problem (CDC 7600) 

Two shift availability would be desirable. 

How do I know if everything is right? Does the motion, 
visual and instrument response correlate with the model 
and pilot input? 

Ljifnpkin/S . Wilson 

Rfucned optimum blend of complexity and fidelity. More 
fidelity, more complexity-^ too much-*- more fidelity might 
compromise operating capability due to increased complexity. 
(Expense increases as well). 

More thought required on products of use of simulation. 

High fidelity, full mission simulation not always required. 
More emphasis on engine model iP" (more than thrust or 
throttle position). (Need to c.^amine critical engineering 
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parameters real-time during simulation) (CDC 7600 will 
have that capacity.) 

4. R. Forrest /V. Lebacqz ' 

- Lack expertise in simulation technology. Need to demon- 
strate and understand simulation capability In order to 
evaluate its affect on research. 

- Need also to translate simulation response characteristics 
(e.g.. Bode plots) into statements concerning the impact of 
simulator cues on the pilot. 

5. E. Aiken 

- Motion system software 

- CGI detail 

- People problems: Too much direction from Sim Sci. 

Need to solicit more research opinion prior to purchase or 
modification of hardware. 

6 . R . Bray 

- Random, insignificant. 

- VMS dynamic performance characteristics questioned by JSC 
people. No easy means available to demonstrate dynamic 
capabilities of VMS. 

7. G. Churchill 

- Need to know simulator system (end-to-end) dynamics 

Motion (see motion) 

Visual (see CGI) effects 
Control system 

- Control system (loaders) 

Need independence of loader natural frequency and damping 
gradient. 

Need acceleration compensation system to compensate 
loader for simulator cab accelerations. 
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